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A survey  of  current  lesson  development  was  considered  essential 


to  determine  the  quality  and  effectiveness  of  instructional  material 


produced  on  the  PLATO  IV  computer-assisted  instruction  system.  The 


trial  period  lasted  for  several  years.  Both  civilian  and  military 


personnel  developed  the  lessons,  in  consultation  with  the  Military 


Training  Centers  Group  at  the  Computer-Based  Education  Research 


Laboratory,  University  of  Illinois,  for  the  Defense  Advanced  Research 

Ai  a - ■ - / t 

Projects  Agency.  ^Eight  representative  lessons j^were  selected  by  a 


Chanute  staff  member.  Then  an  evaluator  prepared  in-depth  reviews 


of  each,  both  in  the  formative  and  summative  stages.  Data  gathered 


from  the  individual  analyses  formed  the  basis  for  the  final  report 


It  indicated  general  trends  and  Isolated  problem  areas  in  the  instruc' 


tional  design.  The  study  resulted  in  an  assessment  of  the  instruc 


tional  effectiveness  and  utilization  of  the  pedagogical  capabilities 


of  the  PLATO  system  in  one  training  environment 
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Introduction 


Background  and  Objectives 

In  writing  this  report,  the  following  preparatory  work  was 
required : 

1.  copies  of  the  selected  lessons  were  made  to  preserve 
them  in  a relatively  fixed  form,  and 

2.  in-depth  reviews  of  each  of  the  eight  lessons  were 
prepared. 

This  version  of  the  lessons  under  evaluation  had  been  in  use  for 
approximately  nine  months  and  were  utilized  by  about  150  students,  all 
military  trainees.  They  are  still  in  use  at  the  present  time.  In 
general,  each  lesson  was  considered  a structural  entity  so  that  the 
significant  aspects  and  component  parts  were  surveyed  as  they  related 
to  the  instructional  whole. 

It  should  be  noted  that  the  MTC  copies  of  the  lessons  represent 
a "snapshot"  in  the  developmental  process.  That  is,  they  were  pre- 
served in  a version  that  has  not  been  subjected  to  intermittent  editing, 
as  the  other  Chanute  lessons  have  been  and  still  are.  Thus,  if  an 
MTC  copy  is  attached  to  one  or  more  Chanute  lessons,  comments  on  the 
latter  must  be  considered  with  the  following  in  mind:  the  reviewer’s 
remarks  about  the  attached  lessons  were  made  as  they  existed  in  the 
period  December  1975  to  February  1976  (see  remarks  in  Appendix  I for 
specifics  on  which  are  active  vs.  fixed  lessons).  Also,  it  is  signi- 
ficant that  "active"  lessons,  (i.e.,  continually  under  revision) 
can  undergo  alterations  of  a moderate  or  an  extensive  nature  at  any 
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time  without  the  reviewer  learning  of  it.  Hence,  some  of  the  features 
and  characteristics  of  the  lessons  surveyed  by  this  report  no  longer 
exist . 

The  goal  of  this  composite  review  is  to  present  a detailed  picture 
of  Chanute  lesson  development  based  on  a select  group  of  lessons. 

The  selection  was  done  by  an  evaluator  stationed  at  Chanute  and  was 
presented  to  the  MTC  group  as  a representative  sampling  of  lessons 

which  spanned  a spectrum  of  validation  dates  from  April  until  August  J 

1975.  (See  Appendix  IV  for  further  details  on  the  exact  dates  of  ; 

particular  lessons  as  well  as  the  actual  Chanute  account  number  of  which  j 

these  lessons  are  earlier  copies.)  The  reviewer  Intends  that  the  ; 

i 

conclusions  made  from  this  survey  will  provide  an  adequate  portrait 

of  the  instructional  design  and  general  appearance  of  these  lessons.  ' 

Method  of  Approach  | 

j 

This  report  will  examine  each  lesson  to  determine  its  success  i 

i 

I 

in  fulfilling  its  own  stated  objective.  Various  specific  cate-  j 

i 

gories  will  be  considered  separately,  since  they  are  crucial  components 
in  the  fulfillment  of  the  lesson  objective.  The  survey  will  not  be 
concerned  with  what  the  lesson  could  have  been  if  conceived  with 
different  teaching  strategies,  author  perspectives,  or  methods. 

Rather,  the  constraints  and  conceptions  of  the  author  and  the  instruc- 
tional system  development  (ISD)  team  will  be  taken  as  a given.  The 
comments  are  presented  with  these  limitations  in  mind. 

On  occasion  the  reviewer  must  comment  on  matters  strictly  outside 


the  discussion  of  the  structure  and  content  of  a particular  lesson, 
i.e.,  the  prior  knowledge  of  the  students  in  relation  to  the  definitions 


3 


of  key  terras  and  concepts.  S ..ace  such  data  are  rarely  available  to 
the  reviewer  (due  to  the  amount  of  time  elapsed  since  the  lessons 
were  prepared,  the  departure  of  authors  from  the  site,  curriculum 
changes,  and  varying  backgrounds  of  new  students),  they  must  be  inferred 
from  the  lesson  itself.  To  assess  the  need  for  definitions,  some  crucial 
questions  must  be  asked.  Are  the  subject  matter  and  terminology 
self-explanatory  or  do  they  demand  prior  knowledge?  If  the  latter, 
was  the  prior  knowledge  included  in  previous  lessons? 

It  is  apparent  that  no  single  study  of  eight  lessons  can  encompass 
the  scope  of  lesson  development  at  a site.  However,  this  limited 
perspective  can  give  some  Indication  of  the  areas  covered  and  the 
problems  encountered.  The  selection  process  employed  attempted  to 
insure  an  overall  survey  of  different  types  of  lessons,  and  the 
report  may  provide  a broad  survey  of  techniques,  albeit  a somewhat 
limited  one  when  it  is  compared  with  the  total  number  of  lessons 
produced.  An  approach  of  this  kind  is  more  feasible  at  this  time 
than  a comprehensive  review  of  every  completed  lesson.  In  addition, 
a draft  of  this  report  was  submitted  to  the  Chanute  staff  for  their 
critique.  All  their  suggestions,  corrections,  and  additions  were 
resolved  or  included  in  this  version.  It  should  also  be  noted  that 
another  report  deals  with  the  process  of  revision  in  the  same  series 
of  lessons. 
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Structural  Analysis 


Procedure 

The  Information  In  this  section  is  largely  drawn  from  the  analysis 
of  detailed  schematic  drawings,  or  flowcharts,  made  of  the  Individual 
lessons.  These  charts  (only  one  of  which  is  included  in  this  report) 
represent  the  way  the  lesson  is  put  together  and  how  it  actually  functions 
when  a student  goes  through  it.  A schematic  of  this  type  provides  the 
following  information  on  the  lesson's  physical  layout: 

1.  the  approximate  length; 

2.  the  presence/absence  of  an  index  or  table  of  contents; 

3.  the  distribution  of  interaction  or  questions  requiring 
responses  from  the  students; 

4.  the  presence/absence  of  forced/optional  review; 

5.  the  individual  modules  that  compose  the  lesson; 

6.  the  distribution  and  quantity  of  the  material  in  the 
modules ; 

7.  the  presence/absence  of  auxiliary  units  used  for  adaptive 
strategies,  student  controlled  branching,  or  additional  help. 

The  study  of  these  drawings  has  facilitated  the  analysis  of  Individual 

lesson  structure.  A segment  of  one  of  the  charts  has  been  included 

(Figure  8 a-b,  pp.  76,  77). 

General  Aspects 

The  implication  of  individualizing  the  lesson  by  allowing  the 
student-controlled  branching  (with  branching  to  the  index  page  as  an 
additional  option)  can  perhaps  be  seen  in  an  analogy.  When  reading  a 
book,  the  student  can  skim  lightly  over  the  material  the  first  time, 
go  through  it  carefully  a second  time,  and  then  review  selected  sections 
for  remedial  work  on  a third  reading.  However,  the  CBE  (computer- 
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based  education)  format  differs  from  that  of  a book:  failure  to  provide 
an  index  with  choice  of  entry  is  akin  to  requiring  a student  to  go 
through  a book  from  beginning  to  end  without  the  option  of  reviewing 
or  skipping  as  determined  by  his  interest.  Even  though  in  some  selected 
cases  there  may  be  one  "best"  way  to  go  through  the  subject  matter, 
student  study  habits  vary  and  it  may  be  reasonable  to  offer  the  option 
of  some  choice.  The  use  of  an  index  might  possibly  lead  to  faster 
completion  times  and  improved  attitudes. 

The  following  can  be  considered  some  of  the  instructional  advan- 
tages of  an  index  with  choice: 

1.  greater  individualization  and  freedom  of  choice  for  the 
student ; 

2.  opportunity  to  review  difficult  concepts  without  the 
danger  of  forgetting  them  due  to  intervening  text 
until  the  student  feels  he  is  adequately  familiar  with 
them; 

3.  opportunity  for  "soft"  or  "selective"  failure — less 
time  needed  for  review,  as  well  as  allowing  the  student 
to  "redeem"  himself  without  officially  failing — an 
important  motivational  factor; 

4.  sense  of  accomplishment  by  student  for  completing  the 

subsection(s) , also  a motivational  factor.  j 

Specifics  ‘ 

Five  of  these  lessons — "starter",  "transmission",  "diesel",  "drive  | 

shaft",  and  "PTO"  offer  a choice  page  for  students  as  they  enter.  The 
remaining  three,  "emission",  "hydraulics",  and  "electricity",  do  not. 

i 

However,  in  the  last  two  lessons,  "hydraulics"  and  "electricity", 
after  the  student  goes  through  the  lessons  once,  he  has  the  option  of 
reviewing  specific  sections  (either  by  his  choice  or  as  selected  by 
the  computer  on  the  basis  of  his  performance  on  the  end  of  lesson 


6 


test. ) 

The  indexed  linear  format  (used  in  the  first  five  lessons  mentioned 
above)  has  the  following  characteristics: 

1.  the  student  can  start  in  the  lesson  at  several  different 
points  (indicated  on  the  choice  page  as  sections), 

but  can  exit  only  at  specified  termination  points  in  the 
sections ; 

2.  within  a section,  movement  is  either  forward  until  reaching 
a termination  point  or  back  to  the  beginning  of  a section; 

3.  at  the  end  of  a section  there  is  a return  to  the  index 
for  a new  choice. 

Thus  there  is  limited  freedom  of  movement  within  this  kind  of  lesson. 

In  lieu  of  such  an  index,  an  outline  of  what  the  lesson  is  to 
cover  has  sometimes  been  included.  Thus,  "electricity"  gives  such  an 
outline  without  the  option  of  entry  point  choice.  This  "roadmap" 
may  be  useful  to  the  student,  but  may  not  be  as  useful  as  a choice 
page.  Each  of  the  sections  is  a separate  aspect  of  the  study  of 
electricity,  and  thus  the  subject  matter  seems  to  be  adaptable  to  a 
breakdown  into  its  component  parts:  electron  theory,  series  circuits, 
parallel  circuits,  series-parallel  circuits,  and  magnetism  and  induc- 
tion. As  the  lesson  stands  now,  without  choice,  it  requires  two  to 
three  hours  at  one  sitting  for  the  student  to  complete.  This  is  in 
contrast  to  the  usual  lesson  completion  time  of  approximately  41 
minutes.  (This  figure  was  computed  from  data  on  all  lessons  supplied 
by  Chanute  listing  average  lesson  completion  times.)  The  length  of 
the  lesson  may  have  caused  at  least  one  student  studying  it  to  request 
multiple  entry  points.^ 

^This  information  was  obtained  during  a visit  to  that  campus. 
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The  subject  matter  expert  is  the  one  to  decide  whether  the 
indexed  linear  format  is  appropriate  for  a specific  lesson.  However,  the 
majority  of  the  eight  lessons  under  review  do  in  fact  offer  this  type 
of  format.  In  some  cases  though,  the  choice  is  not  available  until 
the  student  has  completed  each  section  of  the  lesson:  then  he 
may  review  individual  sections  before  taking  the  final  test.  Though  the  ISD 
principles  practiced  at  Chanute  did  not  provide  for  the  indexed  linear 
format,  the  authors  came  to  an  agreement  with  the  ISD  personnel  and 
included  this  feature. 

In  general,  the  lessons  consist  of  several  moderately  long  seg- 
ments connected  to  an  index:  each  segment  takes  approximately  10 
minutes  to  complete.  Help  units  are  provided  in  three  of  the  lessons: 
"diesel",  "hydraulics",  and  "electricity".  These  auxiliary  units 
consist  of  the  optional  review  of  materials  seen  previously.  In 
addition,  all  the  lessons  make  extensive  use  of  a forced  review  technique, 
making  the  student  re-read  material  whenever  a question  is  answered 
incorrectly  and/or  an  inadequate  score  on  the  end  of  lesson  test  is 
achieved.  (See  Appendix  V for  more  details  on  this  technique.) 

See  Appendix  VI  for  a more  detailed  analysis  of  the  lessons  with 
emphasis  on  student  control  and  individualized  choice  within  the  lesson. 
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General  Conunents 


Components 

It  is  noteworthy  that  an  outline  of  the  lesson  is  accessible  to 
an  author  from  the  objective  page  via  DATA  but  is  not  available  to  a 
student  going  through  the  lesson.  The  outline  is  much  more  detailed 
than  the  brief  one  found  in  the  index.  In  two  lessons  ("drive  shaft" 
and  "electricity")  there  are  no  indications  of  its  existence;  it  was 
discovered  by  an  examination  of  the  coding.  The  reason  the  Chanute  ISD 
staff  gave  for  not  allowing  the  student  access  to  the  teaching  points 
was:  "The  student  will  eventually  see  the  information  there  when 
he  takes  the  lesson."  Nonetheless,  the  teaching  points  may  be  useful 
to  the  student,  and  broadening  their  access  could  at  least  be  investi- 
gated. (For  more  information  on  the  occurrences  of  "teaching  points", 
see  Appendix  III.) 

Organization 

All  lessons  except  one  consist  of  several  modules  (i.e. , sections, 
as  listed  on  the  index  page)  containing  a series  of  coding  units  (i.e., 
basic  building  blocks  of  the  programming  language).  "Emission"  did 
not  use  the  modular  structure;  by  contrast,  it  consisted  of  various 
topics  in  crankcase  ventilation,  pcv  valve  operation,  and  emission 
control.  The  material  in  the  modules  usually  pertains  to  a single 
topic  of  the  more  comprehensive  lesson  subject  matter;  for  example, 
the  planetary  gear  module  contains  the  discussion  of  that  gear  assembly, 
and  that  only  ("transmission"). 

The  modular  arrangement  breaks  up  the  material  into  its  component 
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parts  and  allows  the  student  to  assimilate  it  in  small  increments. 

It  is  an  alternative  to  presenting  an  amorphous  mass  of  material  in 
which  he  may  feel  he  is  adrift,  as  in  "emission".  (See  Appendix  VI.) 
The  student  must  complete  the  whole  lesson  before  he  is  allowed 


to  take  the  end  of  lesson  test,  a practice  which  allows  him  to  become 
exposed  to  all  the  material  in  the  lesson  and  will  give  him  a certain 
familiarity  with  the  subject  matter  covered.  This  exposure  may 
improve  his  chances  for  success  on  the  end  of  lesson  test.  On  the  other 
hand,  since  every  student  sees  every  display,  these  eight  lessons 
employ  only  rudimentary  adaptive  strategies. 

Summary 

In  summary,  the  lessons  have  the  following  organizational  charac- 
teristics: 

1.  the  normal  progression  involves  a pass  through  all  the 
Individual  sections  before  trying  the  end  of  lesson  test 
(except  for  "diesel",  where  it  is  accessible  at  any  time 
from  the  index  page) ; 

2.  the  customary  result  of  a wrong  response  to  a question 
is  being  forced  back  through  one  or  more  of  the  previous 
units  for  remediation  (see  Appendix  VI) ; 

3.  the  usual  consequence  of  an  unsuccessful  attempt  to  pass 
the  test  is  a forced  review  of  selected  sections  in  the 
lesson; 

4.  the  student  cannot  retake  the  test  until  the  indicated 
sections  are  reviewed; 

5.  the  lesson  is  not  considered  completed  until  the  end  of 
lesson  test  has  been  passed  (usually  with  80%  or  better 
accuracy) . 


Every  lesson  objective  was  stated  as  follows: 

Without  reference  identify  the  basic  facts  and  (state)  general 
(operating)  principles  of  with  75%  (80%)  accuracy. 

The  blank  space  contains  the  subject  matter  of  the  particular  lesson 
(e.g.,  automatic  transmissions).  The  objective  is  stated  at  the  begin- 
ning of  each  lesson.  The  student  learns  in  the  end  of  lesson  test 
that  his  "accuracy"  is  shown  by  achieving  the  percentage  score;  when 
he  has  done  so,  he  has  fulfilled  the  author's  pre-stated  objective. 

In  general,  the  lesson  objective  is  :iot  delineated  further. 

No  enabling  objectives  or  sub-objectives  are  specified.  For  each 
lesson  there  is  a three-part  behavioral  objective  which  "Indicates 
what  the  student  should  be  able  to  do  or  say  when  he  has  finished 
the  lesson  . . . the  main  verb  in  a behavioral  objective  is  active" 
(Anderson  & Faust,  1974,  p.  20).  However,  it  is  presented  only  in 
general  terms.  Thus,  it  can  neither  guide  the  student  in  his  study 
of  the  material,  nor  can  it  be  used  by  a subject  matter  specialist 
to  construct  questions  fc  the  end  of  lesson  test.  The  best  that  can 
be  said  of  these  objectives  is  that  they  can  be  considered  statements 
of  the  topics  of  the  lessons  (Merrill,  1974;  Barnes  & Clawson,  1975). 
More  specific  objectives  (e.g.,  "Given  a photograph  of  an  engine. 


the  student  will  locate  the  air  cleaner  on  the  first  attempt"),  if 
used  by  the  author,  have  not  been  provided  for  the  student.  Likewise, 
the  student  is  not  Informed  as  to  what  types  of  performance  criteria 
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(l.e.  , knowledge  of  terminology,  identification  of  components,  listing 
and  knowledge  of  functions)  are  expected  of  him.  In  some  cases,  very 
substantial  subjects  are  addressed  relatively  briefly:  for  example, 
all  knowledge  pertinent  to  the  subject  of  automatic  transmissions  is 
taught  within  an  average  time  of  less  than  50  minutes  ("transmission"). 
However,  according  to  Chanute  personnel,  this  brief  treatment  is  ade- 
quate because  "All  defective  automatic  transmissions  are  shipped  away 
for  repairs." 

For  each  of  these  lessons,  an  objective  could  be  drawn  up  based  on 

the  usual  form  of  the  end  of  lesson  test.  Stated  in  behavioral  (and 

therefore  demonstrable)  terms,  a typical  objective  would  read  as  follows: 

Given  a list  of  10  components  and  12  component  functions 

within  an  automotive  system,  the  student  will 

match  each  of  the  components  with  its  function.  Eight  or 
more  of  the  components  must  be  matched  correctly  on  the 
first  attempt  in  order  to  pass  the  test. 

The  blank  would  contain  the  relevant  subject  matter  of  the  lesson 
(e.g.,  emission  control).  A specific  objective  of  this  kind  has  been 
determined  for  the  majority  of  cases  since  a matching  drill,  end  of 
lesson  test  is  used  in  approximately  3/4  of  the  Chanute  lessons. 

The  author  has  chosen  to  analyze  these  lessons  by  comparing  the 
content  of  the  Individual  lesson  to  the  end  of  lesson  test  rather  than 
by  comparing  the  content  to  the  objectives  or  the  course  control  docu- 
ments. These  documents  and  the  objectives  agree  that  when  the  lesson  is 
completed  the  student's  proficiency  will  be  such  that  he  can  "explain 
[the]  relationship  of  basic  facts  and  state  the  general  principles  about 


i the  subject"  (level  B) . (ISD,  AF  Manual  50-2,  pp.  3-6,  3-7;  see  also 

I 
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lessons  do  not  require  this  level  of  knowledge;  it  is  closer  to  level  A, 
which  requires  only  that  the  student  can  "identify  basic  facts  and  terms 
about  the  subject." 

The  lowering  of  the  knowledge  standards  for  the  vehicle  lessons 
seems  appropriate  for  four  reasons: 

1.  Because  70%  of  the  students'  training  time  is  spent 
in  laboratory  sessions,  many  of  the  "knowledge  items" 

(e.g.,  names  of  parts,  basic  principles  of  operation) 
are  rehearsed  (implicitly)  in  the  course  of  the  labora- 
tory training.  It  is  not  necessary  therefore  that  these 
be  taught  to  a high  degree  of  mastery  in  the  classroom 
(i.e.,  PLATO)  sessions. 

2.  Affective  objectives  are  perceived  by  the  MTC  group  as 
some  of  the  most  important  elements  of  training  (e.g., 
the  Air  Force  philosophy  that  it  may  be  cheaper  to 
replace  than  to  truly  repair,  or  a "go  ahead  and  get 
your  hands  greasy"  attitude).  Since  these  objectives 
may  be  more  easily  achieved  in  the  laboratory  environ- 
ment, it  seems  reasonable  to  limit  the  time  spent  in 
cognitive  areas  which  may  be  less  relevant  to  job 
performance  than  these  affective  outcomes. 

3.  The  performance  of  the  graduates,  based  on  the  field 
evaluation  forms  returned  by  the  supervisors  at  on-the- 
job  training  sites,  indicates  that  the  current  training 
materials  lead  to  satisfactory  job  performance. 

4.  The  knowledge  expected  and  required  of  a graduate  of  the 
course  is  rather  basic.  He  will  rapidly  learn  a great 
deal  from  on-the-job  experiences,  and  he  is  expected  to  be 
dependent  on  more  experienced  staff  Initially. 

The  MTC  author  points  up  the  discrepancy  between  the  materials  in 
order  to  provide  a sound  basis  for  his  decision  to  judge  the  intent  of 
the  lessons  based  on  the  end  of  lesson  tests.  This  is  an  alternative  to 
assuming  that  the  stated  objectives  describe  the  lesson's  purpose.  Aware- 
ness of  this  situation  is  significant.  It  is  then  not  inconsistent  to 
maintain  on  the  one  hand  that  the  lessons  do  not  teach  to  the  objectives 
and  on  the  other  that  they  apparently  are  effective  teaching  instruments. 
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The  general  terminology  used  in  the  objectives  implies  a more 
extensive  treatment  than  the  training  environment  demands.  The  fact 
that  the  lessons  represent  only  an  Introductory  treatment  reflects  the 
lesson  developers'  awareness  of  the  difference  between  the  stated  scope 
of  training  and  that  which  is  actually  required. 

Summary  of  Above 

In  outward  appearance,  the  lesson  consists  of  a series  of  excerpts  from 
a technical  manual  on  the  particular  subject  matter  and  a test  on  selected 
items  at  the  end.  It  is  of  course  open  to  debate  whether  such  breadth  of 
subject  matter  can  be  adequately  covered  given  the  constraint  of  lesson-time 
length.  Though  the  subject  matter  experts  at  Chanute  have  judged  that  this 
amount  of  time  is  sufficient  to  treat  the  material  in  the  depth  needed  for 
their  students,  the  objectives  provide  no  further  guidance  about  the  speci- 
fic facts  to  be  learned  or  the  thoroughness  of  the  training.  Hence,  on  the 
basis  of  the  "teaching  points"  and  the  objectives  available  to  the  reviewer, 
little  more  can  be  said  about  the  match  between  objectives,  test,  and 
instruction.  (See  the  section  on  General  Comments:  Components,  and  Appen- 
dix XI  for  a discussion  of  military  guidelines  for  the  writing  of  objectives.) 
Terminology 

In  general,  there  is  an  appreciable  lack  of  concern  for  defining 
key  terms  and  ideas.  Since  all  the  prerequisites  for  beginning  the 
lessons  were  not  available  to  the  reviewer,  the  students'  familiarity 
with  the  Important  terms  cannot  be  determined  with  certainty.  However, 
definitions  of  these  terms  would  serve  to  refresh  their  memories  if 
they  do  not  have  prior  knowledge  of  the  subject  or  to  bring  them  up 
to  the  same  level  as  more  experienced  students  if  they  are  unfamiliar 


with  the  subject  matter.  From  the  emphasis  given  to  certain 


terms,  it  appears  that  they  are  new  to  the  students  entering  these  lessons. 
Thus,  some  benefits  might  accrue  from  a thorough  discussion  and/or  defini- 
tions of  the  terms. 

This  is  an  important  matter  since  it  is  unlikely  that  authors 
will  be  present  to  provide  assistance  when  the  students  go  through  the 
lessons.  Students  may  be  reluctant  to  seek  help  even  if  a proctor  is 
available  during  the  PLATO/CBE  session.  However,  at  a minimum,  defini- 
tions of  key  terms/concepts  or  even  a question  requiring  the  student  to 
pick  out  a sentence  in  which  the  term/concept  is  used  correctly  would 
tend  to  reduce  uncertainty  or  confusion  resulting  from  lack  of  familiarity 
with  the  terms. 

It  seems  likely  that  the  extra  time  a student  would  spend  reading 
concise  definitions  inserted  into  the  text  would  be  more  than  compensated 
for  by  increased  reading  rates  and  decreased  re-reading  of  difficult 
passages  throughout  the  lesson.  The  author  can  then  confidently  build 
upon  a solid  foundation  as  he  develops  a comprehensive  approach  to  the 
particular  subject  matter.  This  can  be  done  easily  by  putting  all 
key  terms  and  concepts  into  an  auxiliary  unit,  accessible  at  any  time 
via  a branching  routine.  This  reviewer  found  out  only  recently  that 
according  to  Chanute  personnel,  an  attempt  to  buil^  such  a glossary 
was  begun  and  abandoned.  The  reason  for  the  decision  to  abandon  this 
task  is  unknown. 

These  ideas  are  in  no  way  profound  or  novel.  They  are  merely  an 
example  of  a basic  solution  to  a problem  that  is  often  overlooked  during 
the  development  of  a lesson.  Defining  terms  may  not  be  as  stimulating 


as  presenting  a complex  argument  or  graphic  display,  but  it  is  of  vital 
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importance  nonetheless.  The  impact  of  a CBE  presentation  can  be  seriously 
weakened,  if  not  lost  entirely,  if  the  student  fails  to  understand  the 
Important  terms  in  the  lesson. 

Some  examples  of  undefined  terms  or  phrases  may  Illustrate  the 
problem.  The  following  are  in  all  cases  important  to  understanding 
the  relevant  subject  matter  of  the  lesson.  This  list  is  not  exhaustive. 
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Lesson 

a 

emission 

a 

starter 

transmission 

diesel 

hydraulics 
drive  shaft 
PTO 

electricity 


Term  or  Phrase 


vacuum;  non-positive  system,  positive 
system 

(ignition)  switch;  electromagnetic 
switch 

fluid  coupling;  torque  converter; 
servo;  friction-type  clutches 

injector;  glow  plug;  multifuel; 
Roots-type  blower;  TDC 

(majority  of  terms  defined) 

transfer  case;  final  drive  components 

(majority  of  the  few  terms  adequately 
explained) 

(majority  of  terms  defined) 


Except  for  the  two  introductory  lessons,  "hydraulics"  and  "electricity", 
defining  of  key  terms  and  ideas  within  the  lesson  in  which  they  are 
used  is  the  exception  rather  than  the  rule.  Possible  results  of 
undefined  terminology: 

1.  extended  lesson  completion  time  due  to  slower  reading; 

2.  hesitation  to  ask  for  a proctor's  help  or,  alternatively, 
slower  completion  time  due  to  waiting  for  a proctor  to 
attend  to  a question; 

3.  uncertain  attempts  by  the  student  to  •arrive  at  the 
definition  by  himself; 

4.  misconceptions  of  key  terms  or  omission  of  them  to  preclude 
an  impasse. 


^According  to  Chanute  staff,  important  terms  in  these  lessons  were 
defined  in  other  lessons  which  were  prerequisites.  These  latter  were 
not  selected  for  review  in  this  study.  The  retention  of  terms  from 
these  earlier  lessons  is  a subject  for  further  research. 
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Interaction 

The  term  "interaction"  refers  here  only  to  question  and  answer 
interchanges.  The  amount  and  type  to  be  included  in  CBE  are  matters 
of  lively  discussion  (Anderson  & Faust,  1974).  Interaction  is  a funda- 
mental and  unique  capability  of  such  a system:  the  student  can  Interact 
with  the  author  via  the  program  and  the  computer;  in  addition,  the 
student's  understanding  may  be  tested  and  corrected. 

Unlike  lists  of  questions  at  the  end  of  chapters  in  a standard 
textbook,  which  are  really  optional  as  far  as  progress  through  the 
book  is  concerned,  the  student  cannot  proceed  in  most  PLATO  lessons 
until  the  questions  have  been  answered  successfully.  But  the  author 
can  aid  the  student's  progress  by  providing: 

1.  the  correct  answer  after  a pre-determined  number  of 
incorrect  responses  with  the  missed  item(s)  brought 
back  again  at  some  later  point,  or 

2.  additional  information  or  forced  reviews  for  incorrect 
answers. 

A controlling  assumption  in  these  lessons  seems  to  be  that  the 
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student  will  be  aided  in  his  learning  of  the  subject  matter  if  he  must 
recall  what  he  has  just  learned  before  moving  on  to  new  material.  As 
stated  in  the  section  on  Structural  Analysis:  Procedure,  the  flowcharts 
of  these  lessons  indicate  graphically  the  frequency  of  questions.  But 
in  most  cases,  the  discussion  of  terms,  concepts,  and  principles  far 
outweighs  the  number  of  questions  inserted  to  test  recall  and  under- 
standing, as  seen  in  Table  1 . 

It  would  be  possible  here  to  compare  the  quantity  of  questioning 
in  these  lessons  to  a standard,  if  one  existed.  In  any  case,  frame 
after  frame  of  mere  text  tends  to  turn  the  lesson  into  a "page  turner" — 
an  electronic  form  of  the  traditional  textbook.  This  format  does  not 
require  the  complex  hardware  and  higher  cost  of  the  PLATO  system  to 
accomplish  a relatively  simple  teaching  technique.  Likewise,  nearly 
all  the  more  efficient  strategies  by  which  a CBE  system  can  justify 
its  cost  rely  on  substantial  input  from  the  student. 

The  type  of  question  is  important  as  well,  as  seen  in  the  research 
conducted  by  Rothkopf  and  Bisbicos  (1967) . Questions  should  be  of 
some  significance  and  have  a direct  connection  with  the  test  items, 
and  the  lesson  objective  must  always  be  reflected  in  the  questions. 

The  questions  in  these  lessons  should  do  the  following: 

1.  help  the  student  understand  the  material, 

2.  assist  him  in  retaining  the  important  concepts  for 
later  use, 

3.  aid  in  discriminating  between  similar  but  different 
concepts,  and 

4.  in  some  cases,  give  help  in  passing  the  end  of  lesson 
test. 


The  last  is  most  relevant  to  the  author's  objective. 
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The  types  of  interaction  and  the  quantity  may  be  useful  for 
comparative  purposes.  Table  1 provides  some  documentation.  Types  of 
questions  include  fill-in,  multiple  choice,  and  true/false.  The  last 
column  lists  the  number  of  questions  out  of  the  total  number  of  frames 
and  the  percentage  of  questions  to  total  frames. 


Table  1 

Quantity  and  Type  of  Interaction  in  Eight  Chanute  Lessons 


Multiple 

True/ 

Totals 

and 

Lesson 

Fill-in 

choice 

false 

percentage 

emission 

2 

12 

1 

15 

out 

of 

41 

= 

36% 

starter 

13 

33 

0 

45 

out 

of 

95 

= 

48% 

transmission 

0 

14 

11 

25 

out 

of 

43 

= 

58% 

diesel 

8 

6 

0 

14 

out 

of 

74 

= 

19% 

hydraulics 

3 

20 

5 

28 

out 

of 

74 

= 

38% 

drive  shaft 

3 

8 

1 

12 

out 

of 

32 

= 

38% 

PTO 

1 

2 

4 

7 

out 

of 

16 

= 

43% 

electricity 

1 

7 

0 

8 

out 

of 

24 

= 

33% 

The  counts  of  the  numbers  of  questions  and  the  numbers  of  frames 
are  provided  for  the  reader's  use  with  this  caveat:  the  lessons  are 
of  varying  lengths  in  time  and  computer  space  (ECS).  Further,  the 
"attached"  lessons  have  not  been  included  in  the  tallies;  due  to  their 
"non-fixed"  state  they  were  not  broken  down  into  their  component  parts 


as  were  the  "fixed"  lessons.  Hence,  the  absolute  values  are  more 
difficult  to  interpret  than  are  relative  values.  However,  the  lessons 


! 
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sampled  are  representative  of  the  whole  lesson  sequence.  Though  there 
is  some  disagreement  over  what  constitutes  a "frame,"  given  the 
display  and  interactive  capabilities  of  the  PLATO  system,  the  frames 
counted  here  correspond  closely  to  the  number  of  "main  units"  in  the 
lesson. 

The  average  percentage  for  the  eight  lessons  surveyed  is  about 
39%:  one  question  appears  for  about  every  three  frames.  This  figure 
suggests  one  important  characteristic  of  the  lessons:  a large  number 
of  textual  passages  and  the  low  frequency  of  Interaction.  (Sf.® 

Appendix  X also.) 

The  following  are  additional  aspects  of  interaction  in  these  lessons:  i 

1.  the  number  of  distractors  (incorrect  alternative  choices)  i 

is  frequently  less  than  three — thus  improving  the  odds 

of  guessing  the  correct  answer;  !■ 

i 

2.  if  the  student  guesses  correctly,  he  can  avoid  remediation 

and  feedback  in  a substantial  number  of  cases;  ' 

3.  use  of  different  types  of  questions — multiple  choice, 
fill-in,  and  true/false — may  help  to  ascertain  aspects 
or  depth  of  understanding  as  well  as  to  add  variety  or 
maintain  interest. 

The  Chanute  authors  stated  a preference  for  the  multiple  choice 
question  in  the  body  of  the  lessons  reviewed.  They  noted  that  their 
students  had  trouble  with  the  standard  typewriter  keyboard,  considering 
it  a random  arrangement  of  letters  and  other  symbols.  However,  this 
problem  can  be  effectively  minimized  by  using  numbers  located  in  a 
row  at  the  top  of  the  keyboard,  a procedure  which  eliminates  the  need 


to  use  letters.  Incidentally,  the  MTC-supplied  matching  drill  could  have 
been  easily  modified  to  accept  numbers  rather  than  letters.  The 
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alphabet  form  was  chosen  as  standard  by  MTC  because  of  the  relative 
ease  in  utilizing  it  with  more  than  10  possible  answers.  (See  the 
section  on  Specific  Comments:  Tests  and  Appendix  VII.) 

Visual  Presentation 

The  term  "visual  presentation"  usually  calls  to  mind  graphic 
displays — drawings  on  a plasma  screen.  But  there  are  other  considera- 
tions as  well:  margins,  spacing,  presenting  the  question  on  the  same 
frame  as  the  text  Including  the  answer,  and  the  utilization  of  slides 
as  a visual  aid.  In  order  for  each  of  these  to  permit  or  enhance 
learning,  a display  must  be  clearly  explained  and  presented.  The 
following  must  be  taken  into  account:  arrangement  of  text  and/or 
graphics  on  the  screen,  choice  of  type  fonts,  plotting  speed,  and 
drawing  accuracy.  If  the  display  portrays  significant  information,  one 
or  more  questions  about  it  are  appropriate. 

A select  group  of  displays  may  be  illustrative  at  this  point, 
with  commentary  as  to  their  presentation  and  impact. 

"Emission" . Operation  of  a Diverter  Valve. 

A lengthy  period  is  spent  constructing  this  diagram  (about  20  seconds 
due  to  the  complexity  of  drawing  and  use  of  sized  writing)  to  show 
the  operation  of  the  diverter  valve.  Arrows  simulate  movement  of  fluid 
through  it.  The  motion  is  almost  imperceptible,  and  the  impact  is 
thus  rather  insignificant.  (See  Figure  1,  p,  69.) 

"Starter".  Description  of  the  Starting  System. 

The  drawing  is  inadequately  stylized,  composed  of  a square  representing 
a battery  and  a circle  with  a rotating  "x"  inside  to  depict  the  starting 
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motor.  The  drawing  is  of  limited  value  for  representing  the  starting 
system,  the  subject  of  the  lesson.  (See  Figure  2,  p,  70.) 

"Transmission" . Diagram  of  Planetary  Gear  Unit. 

The  gears  of  this  unit  are  NOT  in  constant  mesh,  although  the  accompanying 
text  emphasized  that  constant  meshing  was  always  the  case.  The  drawing 
may  cause  some  confusion  in  the  student's  mind  as  to  what  really  IS 
the  norm.  (See  Figure  3,  p.  71.) 

"Diesel" . Several  allusions  are  made  to  diagrams  in  a handout  that 
was  unavailable  to  the  reviewer.  For  the  student,  this  would  necessi- 

I tate  going  beyond  the  CBE  material  for  help  that  may  not  be  readily 

!! 

i available.  On  the  other  hand,  microfiche  depicting  the  total  comple- 

i ment  of  diesel  components  are  frequently  referenced.  Such  illustra- 

tions are  effective  in  giving  students  a clear  picture  of  the  subject. 

i 

i "Hydraulics".  Cut-away  View  of  a Centrifugal  Pump. 

! 

It  is  a stylized  and  static  display.  Since  the  function  of  a pump 
is  to  move  fluids,  the  center  vanes  could  spin  as  if  fluid  were  flowing 
by.  Moving  arrows  here  could  simulate  motion.  (See  Figure  4,  p.  72.) 

"Drive  shaft".  Several  Drawings  of  the  Power  Train  of  a Vehicle 
showing  the  Post-transmission  Drive  Apparatus  and  its  Chassis. 

These  are  static  displays.  Some  movement  of  the  drive  shaft  in  operation 
might  make  a greater  impact.  (See  Figure  5,  p.  73.) 

"PTO".  Frequent  Graphic  Displays  of  the  Chassis  and  its  Components. 
All  are  fairly  clear  except  for  the  puzzling  presentation  of  the  sprag 
unit  (?),  It  is  difficult  to  determine  what  is  being  depicted  here. 

(See  Figure  6,  p.  74) 
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"Electricity" . Several  Displays  of  the  Structure  of  Various  Atoms,  j 

Complete  with  Protons,  Neutrons,  and  Electrons.  j 

The  frequent  animation  makes  a strong  visual  Impact.  (See  Figure  7,  p.  75.)  | 

These  specific  examples  give  some  feeling  for  the  approach  used  by 
the  individual  authors  in  regard  to  the  use  of  graphics.  Copies  of 
actual  displays  will  be  attached  to  this  report  for  additional  information. 

Graphic  displays  can  thus  be  used  as  a pedagogical  tool  to  offer 
an  explanation  of  material  in  a visual  form.  However,  many  displays 
at  Chanute  were  static.  When  motion  was  attempted,  by  moving  arrows  or 
circles  to  represent  fluids  or  air,  the  animation  was  stilted.  There 

were  either  too  many  cycles  of  movement  or  the  motion  was  lost  amid 

the  larger  display.  "Emission"  contained  examples  of  both  types  of 
ineffective  motion. 

Copy  Frames 

Another  observed  pedagogical  approach  is  the  inclusion  of  both 
text  and  a question  on  that  text  on  the  same  page.  In  the  simplest 

form  of  copy  frame,  all  the  student  need  do  to  answer  the  question  is 

to  look  up,  read  the  answer,  and  write  it  in  the  space  provided. 

Several  examples  of  this  technique  are  found  in  "emission".  Interaction 
is  reduced  to  the  student's  ability  to  search  out  and  copy  the  correct 
answer.  Although,  in  strictest  terms,  copy  frames  are  a form  of 
interaction,  their  effectiveness  as  a learning  device  is  questionable 
(Anderson  & Faust,  1967).  They  stress  visual  recognition  rather  than 
cognitive  processing.  Therefore,  they  are  discussed  in  this  section 


rather  than  in  the  section  on  "interaction." 
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Utilization  of  Color  Slides 

Another  type  of  visual  presentation  is  the  use  of  slides  to 
depict  schematic  diagrams  of  components  and/or  systems  or  photographs 
of  them  in  operation.  A picture  of  the  actual  part  or  a drawing  of  a 
circuit  with  components  added  is  very  effective  for  showing  the  student 
what  to  look  for  when  he  is  on  the  job.  But  for  maximum  impact,  the 
slides  must  be  placed  in  proper  relation  to  the  text.  Improper  super- 
imposition of  slides  and  plasma  display  drastically  reduces  the  legi- 
bility of  the  display,  a condition  especially  apparent  in  the  uses  of  the 
slides  in  "starter".  Superimposed  upon  white  or  light-colored  portions 
of  the  slides,  the  text  is  washed  out  and  virtually  illegible.  The 
strong  back-lighting  from  the  rear  projection  of  the  slide  reduces 
the  background  content.  The  text  could  be  relocated  to  areas  of  greater 
contrast  by  simple  coding  changes. 

Unannotated  presentation  of  a visual  display,  microfiche,  or 
plasma  drawing  is  typically  insufficient  to  insure  that  the  student 
will  learn  the  appropriate  information.  He  must  understand  why  he  is 
looking  at  the  display.  Unless  he  is  given  specific  guidance,  he  may 
be  confused  by  the  amount  of  information  that  graphic  displays  can 
present.  To  insure  comprehension,  one  or  more  questions  on  the  display 
should  be  presented  to  test  the  student's  understanding,  as  is  done 
occasionally  in  "starter". 

Since  a drawing  or  particularly  a photograph  contains  a bewildering 
quantity  of  information,  much  of  which  is  generally  irrelevant  to  the 
author's  objective,  discrimination  betx^een  what  is  important  and  what 
is  marginal  is  often  difficult.  For  example,  the  color  of  paint  on 
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various  automotive  components  may  seem  irrelevant,  but  may  be  part 
of  a color-coding  scheme.  Students  have  been  observed  to  overlook 
information  far  more  obvious  than  the  colors  in  the  display.  The 
author's  job  is  to  guide  the  student's  attention  with  arrows,  circles, 
labels,  and  questions.  (Specific  examples  of  both  kinds — good  and  bad — 
are  found  in  "starter"  and  "diesel".) 

The  following  is  a summary  of  the  previous  comments  on  visual  presentation. 

1.  in  general,  the  visual  displays  are  used  to  explain  the 
subject  matter  of  the  lesson; 

2.  plasma  drawings  are  clear,  very  detailed  and  carefully 
drawn,  but  often  stylized; 

3.  the  use  of  animation  is  ordinarily  appropriate,  but 
could  be  used  to  greater  advantage  (i.e.,  by  using 
more  movement) ; 

4.  the  placement  of  text  on  the  screen  at  the  same  time  as 
the  slide  could  be  changed  to  improve  legibility,  and 

5.  the  explanations  that  accompany  the  visual  displays 
need  to  be  improved. 

(See  also  Dwyer,  1967.) 

Feedback 

In  the  eight  lessons  under  discussion,  there  is  limited  use  of 
feedback,  a term  which  Includes  any  comment  that  appears  after  the 
student  has  typed  in  an  answer  to  a question  and  pressed  NEXT.  Often, 
feedback  is  presented  below  the  line  on  which  the  student  responds. 

Only  a few  questions  are  provided  with  different  types  of  feedback  for 
several  specific  wrong  responses.  In  general,  feedback  does  not 
appear  for  each  question  after  interaction;  even  when  it  is  used, 
it  is  often  similar  to  the  automatic  response  that  PLATO  provides 
(i.e.,  "ok"  or  "no"  after  an  answer  is  judged). 
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Feedback  occurs  In  several  forms.  First  is  the  customary  compliment 
for  a correct  answer,  which  may  influence  the  attitude  and  motivation 
of  the  students.  In  Table  2 this  category  is  labeled  "pos."  The 
compliment  usually  takes  the  form  of  "Well  done!",  "Good!"  or  a similar 
expression.  Because  the  same  response  is  given  regardless  of  the 
number  of  attempts  the  student  needed  to  answer  the  question,  it  may 
at  times  seem  incongrous.  For  example,  after  responding  incorrectly 
to  a multiple  choice  item  three  times,  the  student  finally  answers 
correctly  on  the  fourth  try  and  is  told  "Nice  job!"  After  multiple 
tries  and  final  success,  a simple  "ok"  or  "that's  it"  might  seem  less 
sarcastic. 

The  usual  rejoinder  for  an  incorrect  response  ("no,  try  again"), 
described  in  Table  2 as  category  "neg.",  appears  quite  frequently 
in  the  lessons,  but  its  value  to  a student  is  debatable.  It  does  not 
indicate  where  the  student  went  wrong  nor  in  what  direction  he  should 
go  to  discover  the  right  answer.  Moreover,  PLATO' s answer  judging 
will  automatically  provide  the  simple  "no"  to  incorrect  answers  without 
the  necessity  of  special  coding. 

On  occasion,  the  student  may  even  be  subjected  to  intimidating  or 
abusive  feedback  for  a wrong  response  (see  "antag."  in  Table  2),  i.e., 
"Your  head  is  RIDGED  [sic]  for  selecting  'c'l"  ("transmission").  This 
type  of  feedback  can  decrease  motivation  or  antagonize  the  student, 
and  it  may  be  unfair.  One  can  not  be  certain  that  the  fault  in  giving 
a wrong  response  lies  in  the  student's  understanding  rather  than  in 
the  presentation  of  the  material. 
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In  come  cases,  information  in  the  "feedback"  position  (below  that 
of  the  student  response  on  the  screen)  is  used  to  introduce  new  material 
after  a correct  answer  has  been  given.  This  is  equated  with  "new  mat." 
in  Table  2.  It  can  be  missed  (intentionally  or  unintentionally)  due  to 
a desire  for  rapid  movement  from  one  section  to  the  other.  To  Illustrate, 
some  students  habitually  press  NEXT  twice  after  an  answer  is  given,  with 
the  following  results:  if  they  were  right,  they  move  quickly  to  the 
next  frame;  if  not,  they  are  ready  to  make  another  guess.  Students  have 
even  assumed  that  feedback  for  correct  answers  contained  only  review 
material  (unpublished  study,  Avner,  1970).  Thus,  important  explanatory 
material  may  be  lost  if  placed  in  this  position. 

The  scope  of  material  that  the  feedback  may  include  is  thus 
rather  broad.  It  ranges  from  the  simple  negative  to  the  correct 
answer  itself  ("answer"  in  Table  2).  Neither  extreme  is  likely  to 
benefit  the  student  as  much  as  a middle  course,  offering  hints  or 
additional  Information  to  guide  him  away  from  error  and/or  toward  the 
correct  answer.  A simple  negation  offers  little  help:  he  only  finds 
out  that  he  is  wrong.  Giving  him  the  answer  is  also  of  dubious 
value:  all  he  has  to  do  then  is  type  it  in.  He  may  not  even  understand 
why  it  is  correct  nor  why  it  is  important  for  him  to  retain  it  (if  in 
fact  it  is) . 


The  student  may  want  to  get  the  answer  on  his  own  and  will  not 
be  receptive  to  its  being  given  to  him  without  his  even  requesting  it. 
Of  course  the  answer  should  be  available  after  several  attempts  so  that 
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the  student  can  move  on  and  not  get  hopelessly  mired  in  one  spot 
and  frustrated.  This  is  of  course  very  similar  to  moving  him  on 
automatically  after  he  has  made  several  attempts  to  answer  the  question. 

Both  effect  the  same  result. 

Guidance  toward  a correct  solution  is  another  alternative  (R.  Ander- 
son, 1970;  Lindzey  & Aronson,  1968;  J.  Anderson,  1970)  (described 
as  "help"  in  Table  2).  Given  aid,  he  realizes  he  is  wrong  and  is  put 
on  the  right  path.  But  he  still  has  to  do  some  of  the  detective  work 
in  discovering  for  himself  the  correct  response.  Neither  a bland  rejec- 
tion nor  spoon-feeding  offer  the  satisfaction  that  one  can  gain  by 
searching  out,  with  adequate  supervision,  the  suitable  answer  for  oneself. 
This  is  another  area  where  CAI  can  excel  if  only  the  authors  are 
given  adequate  training  and  time  to  produce  quality  material. 

Drawing  on  the  Navy  document  (Spencer,  Hausser,  et  al,  1975) 
which  lists  the  "rules"  for  effective  feedback,  the  following  summary 
is  presented.  It  discusses  the  various  aspects  of  feedback  and  its 
delivery,  with  examples  drawn  from  the  series  of  lessons  being  consi- 
dered. It  should  be  noted  that  these  are  guidelines  for  "tutorial" 
approaches  and  may  not  be  appropriate  for  simulations. 

Rule  1:  Feedback  should  be  immediate  (within  a few  seconds). 

Observation:  True  for  all  individual  interactions,  not 
true  in  some  cases  for  judging  on  end  of 
lesson  tests.  (See  Appendix  VII  for  specific 
occurrences.) 

Rule  2:  Feedback  should  be  descriptive,  rather  than  evaluati\'e. 

Observation:  The  former  is  true  for  a smaller  percentage  of 

cases  than  the  latter.  , 

Examples;  a.  Descriptive:  "This  Bendix  drive  engages  like  a 
nut  on  a bolt."  ("starter") 

b.  Evaluative:  "I  thought  you  said  you  were  ready." 
("emission") 
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Rule  3:  Feedback  should  be  specific  rather  than  general. 

Observation:  True  in  only  a minority  of  the  cases. 

Examples:  a.  Specific:  "This  starter  uses  the  over- 

running clutch."  ("starter") 

b.  General:  "Try  again"  ("emission") 


Table  2 

Types  of  Feedback 


Lesson 

Pos.^ 

Neg.^ 

Antag. 

New 

Mat.  Answer  Help 

emission 

X 

X 

X 

X 

starter 

X 

X 

X X 

transmission^ 

X 

X 

X 

X X 

diesel'^ 

X 

X 

XXX 

hydraulics 

X 

X 

X 

X 

drive  shaft 

X 

X 

PTO 

X 

X 

X 

electricity 

X 

X 

X 

X 

Note.  An 

"X"  under  a 

heading 

indicates 

that  examples  of  each  kind 

are  found  in  that  lesson, 
a 

This  feedback  is  general  and  not  specific  for  particular  answers. 
The  positive  feedback  (i.e.,  "Good  work")  appears  whether  the  correct 
answer  is  given  on  the  first  or  subsequent  attempts;  it  is  not  related 
to  the  actual  performance  of  the  student. 

The  right  answers  for  the  interaction  are  usually  stressed  by  the 
use  of  bold-face  type  before  the  question  is  asked  (not  on  same  page) . 

c 

The  feedback  for  certain  correct  answers  consists  of  strong  hints 
to  remember  the  information  for  later  use.  See  Appendix  II  for  a 
detailed  treatment  of  use  of  feedback  in  this  lesson. 
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Summary  of  Table  2 

1.  Guidance  or  help  is  provided  in  only  a few  instances 
in  these  lessons; 

2.  The  footnotes  above  indicate  rather  questionable  types 
of  feedback  to  "assist"  the  student; 

3.  the  occurrence  of  general,  non-specific  feedback  (both 
positive  and  negative)  is  widespread  in  these  lessons. 

Tests 

As  was  stated  in  the  section  on' Specific  Comments:  Teaching  to 

I 

an  Objective,  the  end  of  lesson  test  and  lesson  objective  are  rather 
closely  connected.  That  is,  the  objective  is  not  met  until  the  student 
passes  the  test  with  a score  of  75%-80%,  depending  on  the  individual 
lesson.  The  final  test  then  becomes  the  major  focal  point  of  the 
lesson  because  of  the  singular  importance  given  to  it  by  the  lesson 
obj ective. 

The  customary  format  of  the  tests  is  a matching  drill  consisting 
of  one  column  containing  the  functions  and/or  types  of  components 
(usually  10),  and  another  column  listing  the  components/types  (between 
12  and  15).  This  is  the  format  for  the  lessons  on  starters,  transmis- 
sions, diesels,  and  power  take  offs. 

Some  additional  aspects  of  the  matching  drill  are  worth  mentioning. 
In  two  lessons  ("transmission"  and  "PTO")  the  sequence  of  items  in  the 
left  hand  column  (A)  is  not  fixed:  the  descriptions  are  presented 
in  random  order.  Also,  the  directions  are  not  stated  in  any  uniform 
fashion:  often  the  student  is  just  told  to  match  up  the  items  (as 
in  the  lessons  on  starters,  transmissions,  hydraulics,  power  take  offs, 
and  electricity).  Only  in  two  lessons,  "emission"  and  "diesel",  do 


the  directions  point  out  that  one  or  more  items  in  the  left-hand  column 
can  be  used  more  than  once  or  remain  unused.  Thus,  in  the  remaining 
five  cases,  the  student  may  plausibly  conclude  that  he  can  improve 
his  odds  at  guessing  by  eliminating  those  items  used  once. 

The  other  lessons  differ  in  test  format  as  follows. 

"emission":  Five  items  to  be  matched  from  a selection  of  8 

possible . 

"hydraulics";  Two  sets  of  8 items  each  to  be  matched  with 
10  possible  in  each  set. 

"drive  shaft":  Ten  multiple  choice  items,  each  with  3 
distractors. 

"electricity":  Twenty  items  total,  of  which  the  first  7 
are  matching  (8  choices) , and  the  last 
13  are  multiple  choice  with  1 to  3 distrac- 
tors. 

These  data  can  be  summarized  as  follows: 

1.  the  matching  drill  appears  in  the  test  in  all  but  one  of 
the  lessons  in  this  series; 

2.  the  usual  number  of  items  to  be  matched  is  10  from  a 
list  of  12  to  15,  and 

3.  the  usual  sequence  is  to  select  a component  or  type  of 
part/system  from  the  right  hand  column  and  match  it  to 
its  description  in  the  left  hand  column. 

When  a student  selects  either  an  item  to  be  matched  with  its 
function/type  or  one  of  the  multiple  choice  items,  he  presses  NEXT,  but 
no  individual  answer  judging  takes  place.  After  he  proceeds  through 


the  test,  he  is  given  an  opportunity  to  recheck  and  change  or  fill-in 
any  of  the  questions  answered  or  unanswered  on  his  previous  pass,  before 
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of  the  lesson  again  before  retaking  the  test. 

As  part  of  the  testing  procedure,  the  knowledge  of  results  is 
not  usually  given;  however,  in  the  lessons  on  transmissions,  power 
take  off,  and  electricity,  there  is  individual  answer  judging  (i.e., 
feedback  about  which  questions  were  missed,  but  no  feedback  about  why 
the  answers  were  wrong  or  what  answers  were  correct)  when  the  test  is 
scored. 

The  withholding  of  this  individual  judging  in  the  other  lessons 
may  be  due  to  a desire  for  preserving  test  integrity.  It  may  help  to 
insure  that  test  items  are  not  passed  on  from  one  class  to  another. 

In  addition,  if  the  same  test  is  used  when  the  student  retakes  the  test, 
detailed  feedback  on  the  first  pass  might  reduce  the  effectiveness 
of  the  instructional  material  the  student  reviews  before  his  second 
try. 

This  procedure  obviously  contains  some  hazards.  If  the  student 
is  unsure  of  his  answers  and  guesses  on  some  of  them,  he  cannot  tell 
if  his  guesses  are  correct  or  not.  He  may  erroneously  assume  that 
some  answers  were  correct  when  in  fact  they  were  not;  he  may  go  away 
with  faulty  knowledge  of  the  subject  matter  seemingly  confirmed. 

Giving  him  the  option  of  seeing  at  least  the  correct  responses  for  incor- 
rect answers  would  point  out  to  him  where  he  went  astray.  At  the  very 
least,  he  should  not  go  off  with  questions  on  important  parts  of  the 
subject  matter  still  unclear  in  his  mind.  While  the  purpose  of  this 
testing  system  may  be  understandable,  it  does  have  these  undesirable 


side  effects. 
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When  Chanute  evaluators  reviewed  this  section,  they  argued  that  it 
was  unnecessary  to  supply  the  correct  answers  to  a student  who  passed  the 

end  of  lesson  test  but  missed  20%  of  the  items.  Since  the  test  only 

sampled  the  student's  knowledge,  correcting  the  20%  would  therefore 
have  been  only  a small  contribution  to  the  student's  overall  knowledge. 
This  explanation  may  have  been  accurate  but  it  contradicted  the  notion 
of  a criterion-referenced  test,  which  is  what  these  end  of  lesson 
tests  were  designed  to  be. 

One  way  of  avoiding  some  of  these  negative  aspects  can  be  found  in 

the  utilization  of  a matching  test  routine  with  all  its  available 

options.  Most  of  the  end  of  lesson  tests  in  this  series  of  lessons 
employ  some  version  of  this  routine:  it  was  originally  written  for 
Chanute  AFB  (and  other  ARPA  users)  by  the  MTC  group.  This  programming 
device,  used  to  facilitate  the  construction  and  scoring  of  the  hnd  of 
lesson  test,  was  employed  in  several  forms  in  these  lessons.  However,  the 
original  (i.e.,  MTC-supplied)  routine  contained  these  options: 

1.  all  questions  must  be  answered  OR  individual  items 
student  does  not  know  may  be  left  unanswered, 

2.  the  percentage  of  correct  answers  may  or  may  not  be 
shown, 

3.  the  wrong  responses  may  or  may  not  be  indicated, 

4.  the  correct  answers  (for  items  student  has  wrong) 
may  or  may  not  be  shown. 

Explanatory  material  accompanied  the  description  of  the  driver. 

This  material  provided  some  "instructional  guidelines"  to  be  "considered 
when  writing  the  matching  lists"  (Sweeney,  1974,  p.  1): 
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1.  State  clearly  the  relationship  between  the  two  lists 
(i.e.,  tool  to  machine,  part  to  function,  etc.)  . . , 
only  one  relationship  in  any  one  matching  drill  . . . 
[which  should  imply]  a consistent  grammatical  structure 
within  each  list.  (p.  1) 

2.  Use  more  "answers"  than  "questions",  and,  wherever 
possible,  use  the  same  answers  twice  (remembering 
to  tell  the  student  answers  may  be  used  more  than 

once  . . . Students  must  weigh  each  question  separately 
without  using  a "process  of  elimination".  (p.  1) 

The  suggested  guidelines  are  followed  to  a considerable  degree.  For 

specifics  on  the  use  of  directions,  see  Appendix  VII. 

Since  the  original  form  of  the  driver  clearly  contained  the  above 
options  and  directions,  it  must  have  been  decided  (while  other  modifi- 
cations were  made)  not  to  retain  all  the  available  options.  That  is, 
the  options  were  not  missing  because  they  were  difficult  to  program 
or  were  overlooked,  but  were  present  and  purposely  removed  from  the 
original  coding.  (For  further  discussion  of  the  modified  drivers  used 
in  these  lessons,  see  Appendix  VIII.) 


Table  3 

Relationship  of  Test  Items  to  Interaction 


Lesson 


emission 

70% 

stressed 

previously  via 

interaction 

starter 

100% 

tl 

tl  ft 

It 

transmission 

90% 

It 

tt  n 

tt 

diesel 

50% 

tl 

tf  tt 

tl 

hydraulics 

100% 

It 

tt  It 

tt 

drive  shaft 

70% 

II 

!•  It 

tt 

PTO 

40% 

II 

It  tt 

It 

electricity 

85% 

ft 

tl  tl 

tl 
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In  "PTO"  (see  Table  3) , there  is  one  question  not  covered  at  all 
via  interaction  or  text.  Examination  of  the  test  data  for  individual 
items  shows  that  the  performance  of  the  students  on  this  question  was 
about  the  same  as  their  performance  on  other  items:  this  may  indicate 
that  the  students  had  prior  knowledge  of  some  aspects  of  the  subject 
matter  before  beginning  the  lesson. 

Other  comments  related  to  the  testing  procedure  in  these  lessons 

> » 

are  found  in  sections  General  Comments:  Teaching  to  an  Objective  and 

I 

Summary  and  Conclusions. 

Coding 

Errors  in  coding  can  be  of  two  types:  major  or  minor.  The  former 
are  restricted  to  execution  errors  which  terminate  a student's  progress 
in  the  lesson.  The  latter  arise  from  faulty  or  careless  programming" 
these  lead  to  improper  displays  of  text  or  may  cause  minor  problems 
for  the  student  in  the  lesson.  Some  examples  of  these  are  found  in 

Table  4.  i 

I 

! 

A significant  problem  is  the  matter  of  condense  errors.  Though  ] 

invisible  to  the  student,  they  are  clearly  flagged  for  the  author  when  the  | 

lesson  is  selected  for  student  use.  These  errors  can  be  of  various  kinds:  | 

missing  units  or  duplicate  unit  names,  undefined  variables,  and  uninter- 
pretable statements  being  some  of  the  more  common  ones.  In  this  group  of 
lessons,  these  errors  did  not  affect  the  operation  of  the  programming,  but 
they  could  hide  other,  more  serious  errors  (e.g.,  missing  units). 

Some  specific  examples  may  be  illustrative:  "starter"  has  two 
minor  condense  errors,  and  "drive  shaft"  contains  16  condense  errors. 

In  the  latter  lesson,  only  11  appear  on  the  first  page  of  the  condense 
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condense  error  list.  The  second  page  of  the  list,  which  might  contain 
more  serious  problems,  was  not  visible  at  that  time.  Thus,  there  was  a 
danger  in  not  clearing  these  up  as  they  were  pointed  out. 

From  a coding  point  of  view,  the  lesson  can  often  be  Improved  with 
a relatively  small  expenditure  of  time  and  effort.  The  coding  needed 
to  add  specific  feedback  or  a helping  unit  is  relatively  simple 
and  could  be  added  in  a matter  of  minutes.  Implementation  of  such 
changes  can  make  an  operational  lesson  more  useful  for  the  student. 

As  always,  it  is  for  the  author  to  decide  which  of  the  suggestions  he 
wants  to  incorporate  in  his  lesson. 

The  suggested  changes  (indicated  on  the  prints  for  the  examples 
that  appear  later  in  this  section)  are  thus  very  appropriate  for  the 
"lean"  approach  of  lesson  development  advocated  at  Chanute  (Dallman, 
1975;  Misselt,  1975).  It  matters  little  whether  the  coding  is  simple 
or  complex,  unless  the  author  is  producing  a whole  series  of  lessons: 
in  that  case,  sophisticated  coding  will  save  time  and  computer  memory 
space  over  the  long  run. 

Coding  Problems  Encountered  in  these  Lessons 

"Emission" . While  the  student  is  in  the  sections  on  pcv  valve 
and  emission  control,  he  might  hit  the  BACK  key  in  search  of  review 
material  or  inadvertently  by  pressing  the  wrong  key.  Consequently, 
he  will  be  returned  to  the  beginning  of  the  lesson  involuntarily  I 
He  cannot  return  to  his  former  position  in  the  lesson  without  redoing 
all  the  material,  about  20-25  minutes  worth.  A small  change,  Indicated 
on  the  printout  but  not  included  in  this  report,  would  eliminate  this 
interruption  of  normal  progress. 
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"Starter" . Due  to  some  faulty  coding,  part  of  the  section  on 
"Control  circuits"  is  not  functional.  It  omits  part  of  the  discussion 
on  two  types  of  starters,  and  thus  the  student  may  lose  potentially 
valuable  information.  Since  it  occurs  in  the  middle  of  some  text  and 
is  not  a dramatic  break,  the  student  may  not  be  immediately  aware  that 
something  has  been  omitted. 

"Transmission" . There  is  some  minor  non-erasure  of  part  of  the 
feedback  to  a wrong  response  when  NEXT  is  pressed  to  re-try  the  question; 
there  is  also  some  overprinting  of  one  set  of  feedback  over  another. 

"Diesel" . There  is  some-  minor  overprinting  of  lines  of  one  bit 
of  text  onto  the  closing  lines  of  the  preceding  paragraph.  In  one 
instance,  a slide  is  available  via  LAB  but  no  Indication  of  its  presence 
is  given  to  the  student. 

"Hydraulics" . For  one  question  the  feedback  to  a wrong  response 
overprints  the  student  response. 

"Drive  shaft".  On  the  index  page,  if  a student  presses  a key 
designating  any  letter  other  than  the  ones  allowed  (i.e.,  a-e) , an 
execution  error  occurs:  the  student's  progress  through  the  lesson  is 
Interrupted  and  he  is  immediately  deleted. 

"PTO" . A frame  seemingly  designed  as  an  introduction  (as  deter- 
mined both  by  its  content  and  its  name — "intro")  is  in  fact  invisible 
to  all  students  except  those  who  fail  a particular  question.  Then  they 
are  sent  back  to  this  unit  as  part  of  a forced  review. 

One  question  requests  the  student  to  name  two  kinds  of  transfer 
cases  but  the  judging  sequence  will  count  either  half  of  the  answer 
as  completely  correct.  A very  minor  change  will  prevent  this  from 
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occurring. 

A "PRESS  NEXT"  appears  at  the  bottom  of  the  page,  but  a student 
response  must  be  made  before  the  student  can  move  on  from  this  place. 

In  essence,  the  NEXT  key  is  non- functional  at  this  point,  and  the 
directions  car  confuse  the  student. 

"Electricity".  The  student  may  be  confused  by  "DO  NOT  PRESS  NEXT" 
in  large  size  writing  printing  over  "Press  NEXT  to  continue"  in  normal 
size  writing. 


Sunnnarv 


It  Is  of  course  obvious  that  there  are  other  factors  affecting 


the  successful  presentation  of  a CAI  lesson.  Some  of  these  (i.e.. 


the  readability  level,  design  of  end  of  lesson  tests,  etc.)  may  be 


considered  later.  However,  some,  comments  can  be  made  which  sum  up  what 


has  been  presented  in  the  preceding  sections. 


Drawing  upon  the  specific  categories,  the  following  general 


trends  become  apparent. 


1.  The  end  of  lesson  test  cannot  be  attempted  i.ntil  all 
the  individual  sections  have  been  completed  in  every 
lesson  except  "diesel"  ; allowing  it  in  this  one  case 
may  have  been  an  oversight. 


2.  The  format  of  the  stated  objectives  is  identical  in 
all  lessons,  subject  matter  being  the  only  variable. 


3.  For  the  most  part,  dictionary-type  definitions  are  not 
included  for  important  terms/concepts ; rather  the 
components  are  usually  defined  in  terms  of  how  they 
function  within  the  given  mechanism. 


4.  The  number  of  questions  not  directly  related  to  items 
on  the  end  of  lesson  tests  is  small. 


5.  There  is  an  extensive  use  of  visual  displays  depicting 
moving  parts,  which  tend  to  be  static  and  stylized. 


6.  Various  types  of  feedback  are  used  in  these  lessons  with 
different  effects,  but  all  are  an  attempt  to  aid  the 
learning  process. 


7.  With  certain  exceptions,  coding  problems  have  ceased  to 
be  an  important  factor  in  this  series. 


8.  All  lessons  are  adequately  operational. 


Readability  Stud\ 


In  order  for  a student  to  comprehend  a PLATO  lesson  completely, 


his  reading  skill  must  match  or  exceed  the  reading  level  of  the  instruc- 


tion. Although  measurement  of  reading  skills  and  reading  levels  remains 
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imprecise  (particularly  in  the  case  of  technical  instruction  and 
in  situations  where  short  passages  such  as  those  used  in  typical  CBE 
presentations  are  Involved) , several  approaches  have  gained  general 
acceptance. 

One  of  these  has  been  used  for  this  analysis:  the  standard 
definitions  and  procedures  for  readability  of  R.  Flesch  (1949). 

In  the  MTC  study,  in  order  to  speed  processing  and  reduce  computational 
errors,  a computerized  analysis  routine  (contained  in  the  PLATO  lesson 
"evaluate"  written  by  R.  A.  Avner)  was  used  to  convert  raw  data  to 
grade-equivalent  scores.  For  the  eight  lessons  analyzed 

1.  the  mean  grade  level  was  9.0; 

2.  the  highest  grade  level  was  15.0; 

3.  the  lowest  grade  level  was  5.5. 

Although  the  average  reading  level  is  a useful  measure,  perhaps 

even  more  important  is  the  highest  reading  level  of  a lesson.  Because 

of  a "bottleneck"  effect,  the  part  of  a lesson  which  is  most  difficult 

to  read  may  have  the  greatest  impact  on  a student  in  terms  of  his 

frustration  and  his  lesson  completion  time.  The  highest  grade  levels 

can  be  described  as  "easy"  to  "very  difficult"  for  these  students. 

The  range  of  grade  levels  for  each  category  is  given  as  follows: 

very  difficult 
14  and  above 

emission 
electricity 


easy 

8-9.9 


some  difficulty 
10-11.9 


difficult 

12-13.9 


PTO 

drive  shaft 


starter 

hydraulics 


transmission 

diesel 


See  Appendix  XII  for  additional  information  on 


1 

I: 


i 


1.  how  the  reading  study  was  carried  out,  and 

2.  specific  reading  grade  levels  for  the  lessons  analyzed. 
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Conclusions 


The  average  reading  level  of  material  produced  for  students  taking 
technical  training  is  geared  for  the  sixth  grade  level,  in  theory.  In 
practice,  the  grade  equivalents  of  a group  of  students  at  Chanute  ranges 
from  6.8  to  13.6  with  the  average  being  11.78  (AFHRL/ATC  Joint  PLATO  IV 
Evaluation  Report  il>5,  August  1975,  attached  raw  data  sheet);  the  figures 
were  calculated  according  to  the  California  Reading  Achievement  Test 
(Madden  & Tupes,  1966). 

Specifics 

The  following  are  observations  on  particular  lessons. 

"Emission".  The  loose  organization  and  the  failure  of  this  lesson  to 
follow  its  own  objective  may  cause  problems  for  the  students  in  their  attempt 
to  follow  the  thread  of  the  author's  argument.  At  various  points  in 
the  lesson,  the  student  has  the  opportunity  to  review  previously  seen 
material  before  moving  on,  which  is  a limited  form  of  choice.  The  first 
section  of  the  lesson,  on  crankcase  ventilation,  has  extremely  limited 
interaction  and  tends  to  be  a "page-turner"  (i.e. , a lesson  made  up 
of  many  frames  of  text  only) . 

"Starter" . Students  are  likely  to  do  well  on  this  lesson  due  to 
the  stressing  of  test  items  by  interaction.  Within  the  individual 
sections,  there  is  no  roadmap  to  indicate  where  the  student  is  nor 
where  he  is  going:  this  may  lead  to  a feeling  of  loose  organization. 

A substantial  amount  of  the  material  is  presented  with  the  aid  of  slides. 

"Transmission" . There  are  well  thought-out  sequences  of  interaction 


and  complimentary  feedback.  There  is  extensive  use  of  plasma  line  drawings 
and  the  forced  review  technique  to  aid  comprehension.  Names  of  components 
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(i,e.,  SERVO)  appear  as  bold-face,  over-sized  capital  letters. 

"Diesel" . Ostensibly  it  is  an  introductory  treatment  to  the  whole 
subject  of  diesels.  In  fact,  it  does  not  discuss  all  the  aspects  of 
all  types  of  the  compression  ignition  engine,  although  this  might  be 
expected  from  a reading  of  the  objective  (same  as  that  mentioned  in 
the  section  on  objectives).  This  lesson  also  makes  extensive  use  of 
slides  to  display  the  actual  components  of  the  transmission  which  are 
then  described  and/or  their  functions  discussed. 

"Hydraulics" . For  the  most  part,  this  is  a well-planned  and 
well-executed  lesson  that  proceeds  from  a brief  theoretical  discussion 
to  the  practical  aspects  of  hydraulics  in  a balanced  and  organized 
fashion.  Several  review  quizzes  are  employed  to  bring  the  student  up 
to  a certain  performance  level  before  allowing  him  to  proceed  in  the 
lesson. 

"Drive  shaft".  The  limited  feedback  for  a wrong  response  may 
greatly  hamper  or  halt  the  student's  progress,  if  he  does  not  know 
the  correct  answer.  In  general,  the  student  must  reach  a solution  by 
himself  before  moving  on.  This  situation  can  lead  to  frustration  for 
the  student.  Plasma  drawings  are  frequently  used  to  depict  the  various 
components  of  the  automotive  propeller/drive  shaft. 

"PTO" . This  is  a short  lesson,  but  its  very  brevity  might  aid  student 
comprehension,  and  retention  of  the  material  might  help  him  pass  the  end  of 
lesson  test.  In  style,  organization,  and  related  subject  matter  as  well  as 
having  the  same  author,  it  follows  "transmission"  very  closely. 

"Electricity" . Another  example  of  a reasonably  well-prepared 


lesson.  A very  basic  approach  to  the  study  of  electrical  fundamentals 


is  given  in  small  Increments  (e.g.,  matter  -»■  molecules  atoms  -*■  free 
electrons  ->■  current  flow  and  related  matters.)  The  author  tries  to 
tailor  his  material  to  the  student's  needs,  which  is  to  be  commended. 
There  is  extensive  use  of  diagrams  and  interaction  to  stress  important 
concepts  for  understanding  and  retention. 

For  more  information  on  two  other  aspects  of  these  lessons,  use 
of  humor  and  personal  address,  see  Appendix  IX. 


Appendix  I 


The  Lesson  Designations  Explained 


emission  (ratcclO)  = cha3  + some  material  from  cha63 
starter(mtccll)  ■=  cha41 
transmission  (mtccl2)  = cha73 

diesel  (mtccl3)  = cha74  + end  of  lesson  test  in  chal3 
hydraulics  (mtccl4)  = cha78  + some  material  in  cha80/84 
drive  shaft  (mtccl5)  = cha82 
power  take  off  (PTO)  (mtccl6)  = cha86 

electricity  (mtccl8)  = cha97  + other  lessons  listed  in  index  as  sections: 

cha37,  100,  and  43 

Additional  Comments 

1.  The  sequence  of  movement  of  the  student  is  from  the  first 
"cha"  lesson  to  the  other (s)  of  the  attached  lessons;  then  back  again 
to  the  original  "cha"  lesson  for  the  end  of  lesson  test  in  "emission" 
and  "electricity". 

2.  All  the  "cha"  lessons  have  been  exposed  to  continual  editing 
and  revision;  thus,  they  are  not  exact  equivalents  to  their  "mtcc" 
counterparts,  but  they  are  very  similar. 
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Appendix  II 

The  Relationship  of  Interaction,  Feedback,  and  the  Test  Items  in  "diesel" 


This  lesson  was  chosen  for  detailed  study  because  of  the  particular 
way  in  which  test  items  are  stressed  in  the  body  of  the  lesson.  Presen- 
tation of  infonnation  related  to  the  test  items  falls  into  definite 
categories : 

% = percentage  of  students  answering  question  correctly  (79 

cases  total) . 

l/a(dir)  = virtually  identical  questions  in  the  body  of  the  lesson. 

i/a(indir)  = related  question/answer  stressed  by  interaction. 

text  = information  for  answer  provided  in  text,  but  not  emphasized. 


memory 


answer  to  question  provided  in  feedback  to  another  question, 
feedback  to  a question  gives  "Remember  this  for  test". 


blatant  = strong  warning  in  text:  "Don't  forget  (this)"  and/or 

answer  set  off  from  normal  text  via  unusual  positioning 
on  screen  or  special  characters. 

X = one  occurrence  of  this  type  of  stress. 


luestion 


i/a  i/a 

(dir)  (indlr)  text 


Feed- 
back Memory  Blatant 
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Summary  and  Conclusions 


1.  Extra  assistance  is  provided  in  the  form  of  strong 
hints  to  remember  certain  bits  of  data. 

2.  Some  of  the  answers  to  test  items  appear  at  oblique 
places  in  the  lesson  (i.e.,  within  textual  passages, 
in  related  questions). 

3.  Some  are  hinted  at  so  strongly  it  is  as  if  they  were 
stressed  before  the  student  sees  them  on  the  final 
test . 

4.  Two  of  the  questions  (5  and  6)  are  given  very  heavy 
stress  by  the  use  of  strong  hints  ; their  percentage 
correct  figures  may  have  been  Inflated  by  this. 


Appendix  III 
Teaching  Points 
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The  teaching  points  occur  in  all  lessons  and  are  accessible  to 
authors  only  via  DATA.  However,  directions  indicating  their  presence 
and  accessibility  are  not  given  in  all  cases. 


Lesson  Directions  given 


emission 

yes 

starter 

no 

transmission 

no 

diesel 

yes 

hydraulics 

no 

drive  shaft 

no 

PTO 

no 

electricity 

no 

? 


These  explanatory  units  are  noted  since  they  furnish  a detailed  outline 
of  what  is  to  come  in  the  lesson,  although  the  student  himself  never 
sees  the  outline. 


i 

( 
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Appendix  IV 


Criteria  for  Choosing  these  Lessons 


The  lessons  described  In  this  report  were  selected  as  representing 
three  types  of  lessons  grouped  by  validation  date:  lessons  which  met 
the  validation  criterion  1)  very  early,  2)  near  the  median  point,  or 
3)  rather  late  compared  to  other  lessons.  The  Chanute  staff  member 
theorized  that  a later  validation  date  would  indicate  greater  diffi- 
culty in  achieving  that  result.  A lesson  has  validated  when  90%  of  the 
students  going  through  it  get  80%  of  the  questions  correct  on  the  end  of 
lesson  test. 


Lesson 

electricity  (•cha97) 
starter  (cha41) 
emission  (cha3/63) 
diesel  (cha74/13) 

PTO  (cha86) 
transmission  (cha73) 
drive  shaft  (cha82) 
hydraulics  (cha78/80/84) 


Validation  date 
18  August 
18  August 
15  August 
30  June 
25  June 
18  May 
17  April 
14  April 


It  should  be  noted  that  this  list  cannot  be  taken  solely  as  a guide 
to  the  quality  of  the  original  lesson  for  at  least  two  reasons: 

1.  a lesson  that  did  not  validate  could  have  been  set 
aside  for  some  time  before  any  corrections  were  done, 
thus  lengthening  the  time  period  before  it  validated; 

2.  conversely,  another  lesson,  less  likely  to  validate,  could 
have  undergone  extensive  and  more  frequent  remodeling 

and  could  have  validated  sooner. 

Consequently,  this  listing  must  be  used  with  proper  caution  and  not 
be  accepted  at  face  value  as  a reliable  criterion  for  lesson  quality. 


Appendix  V 


The  Forced  Review  Technique 


Because  of  the  frequency  with  which  the  forced  review  occurs, 
it  seems  that  Chanute  designers  adopted  this  strategy  as  standard. 

If  an  incorrect  answer  to  a question  is  given  or  the  end  of  lesson 
test  is  not  passed,  the  usual  procedure  is  to  march  the  student  back 
through  material  he  has  already  seen  so  that  he  will  presumably  get 
it  right  the  second  time  through.  The  technique  may  have  its  advantages. 
It  may  discourage  students  from  guessing  randomly:  such  an  approach 
generally  does  not  produce  genuine  learning.  There  is  a real  danger 
that  such  random  selection  will  occur  since  multiple  choice  and  true/false 
questions  tend  to  predominate  in  these  lessons.  In  addition,  Chanute 
students  taking  early  versions  of  the  tests  were  observed  using  a guessing 
strategy.  Presumably,  this  kind  of  activity  would  be  lessened  when 
students  are  aware  that  incorrect  answers  lead  to  another  pass  through 
the  material. 

The  technique  presents  some  possible  disadvantages.  If  the 
student  had  problems  the  first  time,  there  is  no  guarantee  that  he 
will  not  have  the  same  problems  again.  Then  too,  the  student  may  feel 
that  he  is  being  punished  for  poor  performance  by  this  forced  remediation, 
as  opposed  to  being  given  hints  in  the  feedback  or  extra  help  units. 

In  addition,  in  some  cases  the  student  is  forced  to  go  through  material 
that  he  has  already  covered  successfully  in  order  to  review  subject 
matter  that  he  is  not  adequately  familiar  with.  This  seems  like  a waste 
of  training  time  and  not  in  keeping  with  the  frequently  mentioned  "lean" 
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approach.  In  any  event,  the  Chanute  evaluators  stated  that  the  most 
frequent  specific  complaint  of  students  about  the  lesson  was  a strong 
objection  to  the  forced  review  technique. 

i 


Appendix  VI 

The  Use  of  the  Indexed  Linear  Format 


Name 

Forced  review  of 
whole  lesson  if  final 
test  is  failed 

Index  which  allows 

student  to  choose 

order 

Format  of  the  lesson 

emission 

yes 

none 

Completely  linear. 

starter 

no 

yes 

(at  start) 

All  the  individual  sections 
must  be  completed  before 
the  end  of  lesson  test  can 
be  taken.  Inadequate  perfor- 
mance on  test  leads  to  review 
of  selected  sections.  A 
review  test  is  also  provided 
to  be  taken  before  the  final 
test . 

transmission 

no 

yes 

(one  module 
unlisted) 

All  sections  must  be  com- 
pleted before  taking  test. 

If  test  is  not  passed,  spe- 
cific sections  must  be 
reviewed  before  it  is  retaken. 

diesel 

no 

yes 

Choice  of  sections  and  end  of 
lesson  test  are  available  at 
any  time.  If  test  is  not 
passed,  student  must  go 
through  selected  sections 
again  before  retaking  test. 

hydraulics 

no 

yes 

(at  end) 

No  choice  of  which  sections 
are  to  be  done.  If  test  is 
not  passed  in  course  of  lin- 
ear progression,  there  is  a 
forced  review  of  specific 
sections  (any  or  all  of  12 
are  possible) . 

— 

drive  shaft 

yes 

yes 

. 

If  end  of  lesson  test  not 
passed,  forced  review  of 
whole  lesson 

PTO 

no 

yes 

If  end  of  lesson  test  not 
passed,  selected  sections 
of  the  lesson  must  be 
review  before  test  is 
retaken. 

electricity 

no 

no  choice 

The  student  must  go  through 
all  the  individual  lessons 
that  compose  "electricity" 
before  taking  the  final  test. 
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Appendix  VII 

Directions  for  Taking  the  Test 

The  following  are  the  verbatim  directions  that  a student  sees 
when  he  begins  the  final  test.  Note  the  wide  variation  in  the  quan- 
tity and  content,  fiom  the  very  complete  directions  in  "diesel"  to 
the  omission  of  any  in  "drive  shaft". 


emission 


starter 

transmission 


diesel 


hydraulics 


drive  shaft 


PTO 


electricity 


Complete  the  following  by  matching  the  responses  in 
B,  with  the  statements  in  A.  Some  of  the  responses 
may  apply  to  more  than  one  statement! 

Match  the  components  to  the  proper  statement. 

Match  the  lettered  statement  or  phrase  on  the  right 
with  the  numbered  statement  or  phrase  on  the  left. 

Match  the  components  in  column  B with  the  applicable 
statement (s)  in  column  A.  Type  in  the  letter  of  the 
applicable  components  in  the  blank  space  beside  each 
statement.  REMEMBER:  some  components  may  apply 
to  more  than  one  statement  and  some  components 
may  not  apply  at  all.  To  obtain  a satisfactory 
rating  you  must  select  eight  of  ten  correctly. 

Match  the  applicable  lettered  statement  or  phrase 
on  the  right  with  the  numbered  statement  or  phrase 
on  the  left. 

(No  directions  stated  for  test — 10  multiple  choice 
questions. ) 

Match  the  lettered  statement  or  phrase  on  the  right 
with  the  numbered  statement  or  phrase  on  the  left. 

Match  the  following  statements:  (4  item  matching 
drill) . 

Match  the  name  to  the  circuit:  (3  diagrams  of 
circuits  to  be  identified). 

Type  in  the  letter  of  the  correct  response:  (13 
multiple  choice  questions) . 


A 
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Lesson 

emission 

starter 

transmission 

diesel 

hydraulics 
drive  shaft 
PTO 

electricity 


Appendix  VIII 

Utilization  of  an  KTC-Supplled  Matching  Test  Routine 


Options  Utilized 

No  judging  of  individual  answers  after  each  answer 
is  selected,  but  a percentage  of  correct  answers 
is  given  at  the  end. 

Same  comment  as  above. 

Both  correct  and  wrong  responses  indicated  after 
test  is  graded,  but  no  correct  answer  feedback  is 
provided. 

No  individual  judging  of  answe  after  each  answer 
is  selected,  but  a percentage  Oi.  correct  answers  is 
given  at  the  end. 

Same  as  above. 

Same  as  above. 

Individual  answer  judging  with  option  to  review 
Incorrect  answers  is  provided  at  the  end  of  the  quiz. 

No  individual  answer  judging  is  provided  after  each 
selection,  but  a percentage  of  correct  answers  is 
given  at  the  end,  and  there  is  an  option  to  go  back 
through  the  ones  missed  for  review. 


Appendix  IX 


Humor  and  Personal  Address 


CBE  lessons  often  employ  techniques  to  Improve  student  motivation 
such  as  using  humor  and  addressing  the  student  by  name  or  nickname. 
Some  selected  examples  follow  with  commentary. 


transmission 


diesel 


drive  shaft 


Student  is  addressed  by  name  once.  Possible  attempts 
at  humor  (i.e.,  "That's  what  I call  auto  screen 
change!")  at  beginning  of  lesson.  Also  some  heavy 
handed  humor  is  found  in  feedback  (i.e.,  "You 
have  a SPLINED  mind  ..."  for  selecting  the  wrong 
response) . 

Possible  attempt  at  humor  (little  figure  representing 
"air").  Also,  there  is  a greeting  at  beginning 
of  lesson:  "good  morning,  afternoon,"  etc.  depending 
on  time  of  day  that  student  signs  in. 

Students  are  addressed  by  name.  A possible  attempt 
at  humor  (i.e.,  "wait  here  to  get  the  shaft"). 


electricity  Good  attempt  at  humor  with  portrayal  of  B.  Franklin, 

lightning,  and  kite  as  an  introductory  graphic  to 
the  lesson. 


Appendix  X 


Types  of  Questions  Used  and  Conunents 


Lesson 

Kind  of  Question 

Comments 

emission 

Simple  fill-in  (with 
answer  on  same  page 
and  multiple  choice. 

Only  two  distractors 
in  most  multiple  choice. 

starter 

Large  quantity  of 
questions  of  differ- 
ent types. 

Only  two  distractors 
in  many  cases. 

transmission 

Many  true/false 
although  the 
suitability  of 
subject  for  this 
type  cannot  be 
ascertained. 

The  multiple  choice 
only  have  two 
distractors  which  may 
encourage  guessing. 

diesel 

Elementary  level, 
("What  is  compres- 
sion ignition  eng- 
ine?" Ans.  "diesel.") 

Very  plausible  distractors. 

hydraulics 

Thought-provoking 
multiple  choice. 

Very  plausible  distrac- 
tors. 

drive  shaft 

Several  questions 
of  true/false  kind 
are  very  similar  to 
each  other. 

Confusing  statements 
in  the  true/false  ques- 
tions. 

PTO 

Varied  as  to  type; 
all  three  (fill-in, 
multiple  choice,  and 
true/ false)  are  used. 

Substantial  amount  of 
interaction  even  for 
short  lesson. 

electricity 

Interspersing  of  text 
and  questions  of  all 
types  (true/false, 
multiple  choice,  and 
fill-in)  is  well- 
balanced. 

Only  2-3  distractors 
in  several  questions. 

n 


Appendix  XI 

Writing  of  Objectives  discussed  In  Air  Force  Documents 

Guidelines  from  a Chanute  AFB  document^  conclude  that  non- 
specific objectives  obligate  the  author  to  teach  all  of  the  information 
needed  within  the  particular  area  of  the  objective  (p.  72).  Objectives 
similar  to  those  used  by  Chanute  PLATO  project  staff  were  given  as 
examples  of  those  which  are  "not  properly  written"  (p.  71):  e.g., 

"TASK,  repair  a flat  tire." 


For  some  discussion  of  these,  see  the  following  sections  in 
Application  of  the  Systems  Approach  to  Training.  Chanute  Air  Force 
Base,  Illinois:  Department  of  Weapons  System  Support  Training, 

January  1972. 

Sec.  5 "Task-subject  analysis,"  pp.  57-86. 

Sec.  6 "Preparation  of  statements  of  learning  objectives," 
pp.  87-100. 

Sec.  7 "Development  of  learning  objectives,"  pp.  101-110. 

See  also  the  more  general  discussion  of  writing  objectives  in  Chapters 
1 and  2 of  Handbook  for  Designers  of  Instructional  Systems,  Vol.  Ill, 
"Objectives  and  Tests,"  (AFP  50-58).  Washington,  D.C.:  Department  of 
the  Air  Force,  15  July  1973. 
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Appendix  XII 

Chanute  Readability  Study 

Only  text  which  presented  new  Information  was  included  in  the 
analysis.  Test  directions,  feedback,  labels  on  graphics,  interaction 
and  test  questions  were  excluded. 

1.  Sample  passages  containing  100  words  were  selected.  In 
some  cases,  three  samples  were  used;  one  from  the  beginning, 
middle,  and  end  of  the  lesson.  If  the  average  fell 

within  a range  of  1.5  grade  levels  (i.e.,  7.0  to  8.5), 
no  more  samples  were  taken.  If  a larger  range  was  found, 
more  samples  were  taken  to  insure  reliability.  In  some 
cases  however,  five  samples  exhausted  the  number  of 
usable  passages  within  the  lesson. 

2.  Counting  was  started  at  the  beginning  of  a frame  and 
continued  in  100  consecutive  word  sections.  (Hyphenated 
words,  contractions,  and  abbreviations  counted  as  one  word; 
symbols  and  dates  or  other  numbers  considered  as  a unit 
were  counted  as  one  word.) 

3.  The  number  of  syllables  in  each  passage  of  100  con- 
secutive words  was  counted. 

4.  The  number  of  complete  sentences  nearest  the  100th  word 
was  counted.  (For  example,  if  sentence  //5  stopped  at 
word  98,  the  average  was  98/5  words  per  sentence.) 

Since  several  passages  were  analyzed,  an  approximate  range  of  the 

reading  levels  of  the  lesson  has  been  calculated. 


r 
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Lesson 

//  Samples 

Range 

grade  level 

Mean 

grade  level 

High 

Low 

c 

emission 

7 

14.0 

7.0 

9.5 

starter 

9 

11.5 

5.5 

9.l‘’ 

transmission 

6 

13.0 

7.0 

9.4 

diesel 

8 

13.0 

6.0 

9,4 

hydraulics'^ 

10 

11.5 

7.5 

9,1 

drive  shaft 

4 

9.0 

6.5 

7.5 

PTO 

3 

8.0 

7.5 

7.7 

Q 

electricity 

14 

15.0 

7.0 

10.2^ 

The  figures  in  these  two  columns  are  based  on  selected  sample 
passages.  The  limits  of  the  Flesch  system  aie  5.5-15.0.  Anything 
oelow  or  above  those  limits  is  given  the  value  of  the  nearest  limit. 

^These  values  are  only  approximate  due  to  limitations  in  range  of 
possible  reading  scores. 

Also  includes  cha63. 
d 

Also  includes  chaSO. 
e 

Also  includes  cha37,  cha43,  chalOO. 
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Figure  2 

Description  of  Starting  System 


cranking  — »■ 
motor 


— botfery's  stored 
chemicol  (electricol) 
energy 


Figure  4 


Cut-away  View  of  Centrifugal  Pump 


positive 

o 


non -positive 
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Figure  6 

Display  of  Sprag  Unit 
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Flowchart  of  "transmission 


Figure  8b 

Detail  of  Module  #1  of  "transmission" 
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